<html>
<head><meta charset="utf-8"><title>Bors is stuck · t-infra · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/index.html">t-infra</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html">Bors is stuck</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="238847954"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238847954" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238847954">(May 15 2021 at 00:43)</a>:</h4>
<p>Bors is stuck: <a href="https://bors.rust-lang.org/queue/rust">https://bors.rust-lang.org/queue/rust</a></p>



<a name="238848154"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238848154" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238848154">(May 15 2021 at 00:46)</a>:</h4>
<p>investigating</p>



<a name="238848324"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238848324" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238848324">(May 15 2021 at 00:49)</a>:</h4>
<div class="codehilite"><pre><span></span><code>error: Your local changes to the following files would be overwritten by checkout:
    2021-05-14T20:46:41.710-04:00   src/bootstrap/builder.rs
    2021-05-14T20:46:41.710-04:00   src/bootstrap/check.rs
    2021-05-14T20:46:41.710-04:00   src/bootstrap/test.rs
    2021-05-14T20:46:41.710-04:00   Please commit your changes or stash them before you switch branches.
</code></pre></div>



<a name="238848482"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238848482" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238848482">(May 15 2021 at 00:51)</a>:</h4>
<p>on <a href="https://github.com/rust-lang/rust/commit/26141c72766471eae406031cba22656672bb63bb">https://github.com/rust-lang/rust/commit/26141c72766471eae406031cba22656672bb63bb</a> being merged into auto</p>



<a name="238848582"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238848582" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238848582">(May 15 2021 at 00:53)</a>:</h4>
<p>my guess is the cache somehow got messed up, going to try to figure out how</p>



<a name="238848614"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238848614" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238848614">(May 15 2021 at 00:54)</a>:</h4>
<p>seems to be ok at 2021-05-14T18:04:48.901-04:00 Switched to and reset branch 'homu-tmp'</p>



<a name="238848677"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238848677" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238848677">(May 15 2021 at 00:54)</a>:</h4>
<p>broken at 2021-05-14T18:07:00.843-04:00 error: Your local changes to the following files would be overwritten by checkout</p>



<a name="238848734"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238848734" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238848734">(May 15 2021 at 00:55)</a>:</h4>
<p>hm, nothing suspicious in between</p>



<a name="238848736"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238848736" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238848736">(May 15 2021 at 00:55)</a>:</h4>
<p>not really sure what's up</p>



<a name="238848738"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238848738" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238848738">(May 15 2021 at 00:55)</a>:</h4>
<p>going to restart bors</p>



<a name="238850291"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238850291" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238850291">(May 15 2021 at 01:01)</a>:</h4>
<p>ok, seems to have gotten unstuck</p>



<a name="238850293"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238850293" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238850293">(May 15 2021 at 01:01)</a>:</h4>
<p>not sure what happened</p>



<a name="238850423"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238850423" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238850423">(May 15 2021 at 01:02)</a>:</h4>
<p>although now it's testing.. twice on <a href="https://github.com/rust-lang/rust/pull/84876">https://github.com/rust-lang/rust/pull/84876</a></p>



<a name="238850450"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238850450" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238850450">(May 15 2021 at 01:02)</a>:</h4>
<p>hopefully that's ok</p>



<a name="238856753"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238856753" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Aaron Hill <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238856753">(May 15 2021 at 01:34)</a>:</h4>
<p>It's stuck again</p>



<a name="238858086"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238858086" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238858086">(May 15 2021 at 01:55)</a>:</h4>
<p>Hopefully the bors repo isn't corrupted by <a href="https://github.com/rust-lang/rust/issues/82208">#82208</a>.  I'm uncertain if bors reuses a local clone (I assume it does).  git is spitting out <code>error: add_cacheinfo failed to refresh for path</code> which from what I can surmise as an internal error.  Perhaps things aren't getting cleaned up correctly?</p>



<a name="238858268"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238858268" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238858268">(May 15 2021 at 01:58)</a>:</h4>
<p>And to clarify, when it hits that error, the merge has partially succeeded, and leaves the repo in a messy state that requires a reset.</p>



<a name="238859320"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238859320" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238859320">(May 15 2021 at 02:14)</a>:</h4>
<p>Hmm, I don't see anything in <code>homu</code> that deletes the repo or calls <code>reset --hard</code>.  I'm suspecting that there were some modified files, and then the next merge attempted to modify them.</p>



<a name="238859329"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238859329" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238859329">(May 15 2021 at 02:14)</a>:</h4>
<p>After the failure I have the following modified files left in the worktree:</p>
<div class="codehilite"><pre><span></span><code>    modified:   .gitmodules
    modified:   Cargo.lock
    modified:   src/bootstrap/builder.rs
    modified:   src/bootstrap/check.rs
    modified:   src/bootstrap/test.rs
    modified:   src/bootstrap/tool.rs
    modified:   src/bootstrap/toolstate.rs
    modified:   src/ci/docker/host-x86_64/x86_64-gnu-tools/checktools.sh
    modified:   src/ci/scripts/should-skip-this.sh
</code></pre></div>



<a name="238859827"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238859827" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238859827">(May 15 2021 at 02:22)</a>:</h4>
<p>Yeah that seems really plausible</p>



<a name="238859852"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238859852" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238859852">(May 15 2021 at 02:23)</a>:</h4>
<p>Not sure how much can be done...</p>



<a name="238860986"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238860986" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Eric Huss <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238860986">(May 15 2021 at 02:40)</a>:</h4>
<p>I think just deleting the <code>$cache_dir/rust-lang/rust</code> directory should unstick it.  And maybe ask caleb to hold off pushing changes to the rustfmt PR unless the issue is fixed.</p>



<a name="238863196"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238863196" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238863196">(May 15 2021 at 03:15)</a>:</h4>
<p>I'm not sure that's "just" unfortunately</p>



<a name="238863262"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238863262" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238863262">(May 15 2021 at 03:16)</a>:</h4>
<p>I think I'll close the PR and restart bors; I think it gets cleared out on restart, though not confident.</p>



<a name="238863879"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238863879" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238863879">(May 15 2021 at 03:26)</a>:</h4>
<p>ok, closed PR and restarted bors</p>



<a name="238864006"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238864006" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238864006">(May 15 2021 at 03:29)</a>:</h4>
<p>ok hopefully that worked</p>



<a name="238864049"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238864049" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238864049">(May 15 2021 at 03:30)</a>:</h4>
<p>alright, i'm off, will investigate further tomorrow</p>



<a name="238883211"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238883211" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238883211">(May 15 2021 at 09:18)</a>:</h4>
<p><span class="user-mention silent" data-user-id="120518">Eric Huss</span> <a href="#narrow/stream/242791-t-infra/topic/Bors.20is.20stuck/near/238860986">said</a>:</p>
<blockquote>
<p>I think just deleting the <code>$cache_dir/rust-lang/rust</code> directory should unstick it.  And maybe ask caleb to hold off pushing changes to the rustfmt PR unless the issue is fixed.</p>
</blockquote>
<p><span class="user-mention silent" data-user-id="120518">Eric Huss</span> <a href="#narrow/stream/242791-t-infra/topic/Bors.20is.20stuck/near/238858086">said</a>:</p>
<blockquote>
<p>Hopefully the bors repo isn't corrupted by <a href="https://github.com/rust-lang/rust/issues/82208">#82208</a>.  I'm uncertain if bors reuses a local clone (I assume it does).  git is spitting out <code>error: add_cacheinfo failed to refresh for path</code> which from what I can surmise as an internal error.  Perhaps things aren't getting cleaned up correctly?</p>
</blockquote>
<p>the cache dir should be cleared when bors restarts btw</p>



<a name="238883218"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238883218" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238883218">(May 15 2021 at 09:18)</a>:</h4>
<p>it's not stored on efs iirc</p>



<a name="238891098"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238891098" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238891098">(May 15 2021 at 11:35)</a>:</h4>
<p>Yeah based on logs it isn't, though that does make it impossible to inspect</p>



<a name="238909700"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238909700" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238909700">(May 15 2021 at 16:36)</a>:</h4>
<p><span class="user-mention" data-user-id="116122">@simulacrum</span> pretty sure it should be possible to log into the fargate container</p>



<a name="238909794"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238909794" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238909794">(May 15 2021 at 16:38)</a>:</h4>
<p><a href="https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-exec.html">https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-exec.html</a></p>



<a name="238910169"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238910169" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> simulacrum <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238910169">(May 15 2021 at 16:44)</a>:</h4>
<p>Huh, didn't know - that seems really useful.</p>



<a name="238910223"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/242791-t-infra/topic/Bors%20is%20stuck/near/238910223" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Pietro Albini <a href="https://rust-lang.github.io/zulip_archive/stream/242791-t-infra/topic/Bors.20is.20stuck.html#238910223">(May 15 2021 at 16:45)</a>:</h4>
<p>they added it like a couple of months ago, was really happy when I saw it</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>